Virtual Face Image Generation For Illumination And Pose Insensitive Face Recognition 

Wen Gao u . Skiguang Shan', Xiujuan Chai 2 , Xiaowei Fu 2 
Insutute of Computing Technology, CAS, P.O.Box 2704, Beijing, China, 100080 
Department of Computer Science, Harbin Institute of Technology, Harbin, China 150001 



ABSTRACT 

Face recognition has attracted much attention in the past 
decades for its wide potential applications. Much progress 
has been made in the past few years. However, specialized 
evaluation of the state-of-the-art of both academic 
algorithms and commercial systems illustrates that the 
performance of most current recognition technologies 
degrades significantly due to the variations of illumination 
and/or pose. To solve these problems, providing multiple 
training samples to the recognition system is a rational 
choice. However, enough samples are not always available 
for many practical applications. It is an alternative to 
augment the framing set by generating virtual views from 
one single fece image, that is, relighting the given face 
images or synthesize novel views of the given face. Based 
on this strategy, this paper presents some attempts by 
presenting a ratio-image based face relighting method and 
a race re-rotating approached based on linear shape 
prediction and image warp. To evaluate the effect of the 
additional virtual face images, primary experiments are 
conducted using our face specific subspace method as face 
recognition approach, which shows impressive 
improvement compared with standard benchmark face 



1. INTRODUCTION 

Face recognition has attracted much attention in the past 
decades for its wide potential applications in commerce 
and law enforcement, such as raug-shot database matching, 
identity authentication, access control, information 
security, and surveillance. Much progress has been made 
in the past few years [1,2]. 

Since the 1990s, appearance based methods have been 
dominant researches, from which two FRT categories wen- 
derived: holistic appearance feature based and analytic 
local feature based. Popular methods belonging to the 
former paradigm include Eigenface[3], Fisherface[4] 
Local Feature Analysis (LFA)[5] and Elastic Bunch Graph 
Matching (EBGM)[6] are typical instances of the latter 
category. In recent years, Eigenface, Fisherface, EBGM, 
Active Shape Models and Active Appearance Model 
(ASM/AAM)[7, 22], subspace discrimination analysis[8] 
and SVM[10] based approaches have attracted much 
attention. FERET evaluation has provided extensive 
comparisons of these algorithms [9]. 



However, face recognition remains a difficult, unsolved 
problem in general. The performance of almost all current 
face recognition systems, both best academic systems and 
most successful commercial systems, is heavily subject to 
the variations in the imaging conditions. It has been 
discovered by the FERET and FRVT test that pose and 
illumination variations are among the several bottlenecks 
for a practical face recognition system [9]. By far, no 
revolutionary practical solutions are available for these 
problems. However, some solutions to pose and 
illumination problems do have emerged including 
invariant feature based methods [16], 3D linear 
illumination subspace [4], linear object class [11], 
illumination and pose manifold [12], Symmetric Shape- 
From-Shading [8], photometric alignment [13], Quotient 
Image [14J, illumination cones [15], Lambertian 
Reflectance and Linear Subspace [17], Eigen light-fields 
[18] and parametric linear subspace [ 1 9]. 

Generally, we may categorize approaches used to cope 
with variation in appearance into three kinds: invariant 
features, canonical forms, and variation modeling [20]. 

The first approach seeks to utilize features that are 
invariant to the changes in appearance. Examples of such 
representation considered by early researchers are edge 
maps, image intensity derivatives, and images convolved 
with 2D Gabor-like filers. However, Adini's empirical 
study had shown that "None of the representations 
considered is sufficient by itself to overcome image 
variations because of a change in the direction of 
illumination"! 16]. Most recently, the Quotient Image [14] 
is reported to be invariant to illumination and may be used 
to recognize faces when lighting conditions change. 

The second approach attempts to "normalize" away the 
variation in appearance, either by image transformations or 
by synthesizing a new image from the given image in some 
canonical form. Recognition is then performed using this 
canonical form. Examples of this approach include [8. 21]. 

The idea of third approach, variation modeling, is to 
learn, in some suitable subspace/manifold, the extent of 
the variation in that space/monifold. Recognition is then 
conducted by choosing the subspace/manifold closest to 
the novel image. Currently, this paradigm has been 
recognized as the dominant one among the three 
approaches! 1 1, 1 2, 1 3, 1 5, 1 7, 19, 20], 

In this paper, we investigate the possibility to augmeni 
the training set for modeling the variations by generating 
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virtual face images when changing lighting conditions or 
viewpoints. Th,s is especially useful for applications that 
only limited samples per face are available for training. 

The paper is organized as: In section 2 wc first 
described briefly our works on ASM for aligning face 
images. Section 3 describes the ratio-image based face 
relighting approach, followed by virtual view prediction 
based on shape prediction. Our recognition approach 
based on Face Specific Subspaces (FSS) is presented in 
section 5. Experiments are set up in the last section. 
2. Our Works On Feature Correspondence 
Both our lace relighting method and fece rotating 
method need accurate feature correspondence. Therefore, 
we first describe our works on feature extraction briefly. 
Refer to [23], [24], [25] respectively for details of our 
work on face segmentation, eye localization and face 
shape extraction. Our face detection method, named Face 
Cemer-of-Gravity Template, is based on some 
observations on the configure relationship between major 
face organs. The eyes are then localized by growing a 
region window from the approximate center of the 
detected face and checking its characteristics. After eyes 
are located, we attempt to combine the ASM's local 
texture models and AAM's global appearance models for 
spare facial feature correspondence. To integrate the local 
profile and global appearance constraints, the subspace 
reconstruction residual of the global texture is exploited to 
evaluate the fitting degree of the current model to the 
novel image. And, similar to the AAMs, global texture is 
used to predict and tune the model parameters. Some 
results of o ur feature extrac tion method areshown in Fig. I 




e Relighting for Modeling 
Illumination Variations 

In this section, a ratio image based face relighting 
method is presented. The method is based on the 
assumptions that any face were a convex surface with a 
Lambettian function, that is, a face image can be described 
by the product of the albedo and the cosine angle between 
a point light source and the surface normal: 

I (x,y) = p(x,y)n(x,y)s 
where p(x,y) is the albedo associated with point in the 
image, n(x, y) is the surface normal direction associated 



with point x,y in the image, and s is the point light source 
direction and whose magnitude is the light source intensity 
Thus, our problem can be formulated as: Given a face 
image I„ under normal light source, J 0 , we need to relight 
the face under other light sources, e.g. 5?,. To solve this 



"""-"^ " muu-image oasca method. 

First, we define the ratio-image for the i" face (person) 
under the r light source as: 

»*.-/»//« 

= (A«/?*)/(A«/-4) 

= {n, ■ s t )/(«, -s 0 ) 
where p, is the albedo {surface reflectance) associated 
with the f face, n,.is the corresponding surface normal 
directions, and s a and s, are the standard and target 
point light source directions respectively. Thus, we have: 

A* = PA s k =(p,n i -s 0 )®r ik =I m ®r lk . 
where ® denotes Cartesian product. This means that, 
given the ratio-image and the standard face.image, we can 
relight the face to the jf* light source. 

The ratio-image above defined is almost useless since it 
is only applicable to the r* face. However, notice that all 
faces have similar 2D and 3D shapes, so we can try to first 
warp all feces to the same shape and then compute the 
ratio-image for relighting the standard face images. It is 
then easy to reverse warp the relit face image back to its 
original shape. Currently, we just warp the 2D fece image 
to a predefined mean shape, as defined in ASM. After the 
warp procedure, all face images are expected to have quite 
similar 3D shape. Therefore, given a training set; by warp 
all the face images under different lighting conditions to 
the same shape, we can define the universal ratio-image 
for the k light source as the mean of all the specific ratio- 
image of each face in the training set: 

where N is the total faces in the training set, r a is the 
shape-free texture warped from the f face images lighted 
under the V light source, and T IB is its corresponding 
texture under the standard light source. 

By varying the different light source, we can get the 
ratio-image for each light source and/or combine them to 
ratio-image for any lighting conditions. 

After the ratio-images are computed, any novel face 
image /„ with standard lighting conditions can be relit by 
the following procedure: 

1. Find the face in the picture and extract its 2D face 
shape using the method descried in Section 2; 

2. Warp the face to texture T 0 according to the 
predefined mean shape; 



111-150 



3. Relight the face image under *-th lighting condition 
according to the A--th ratio-image by: 

4. Reverse-warp the texture 7} to its original shape to get 
the relit image J k .; 

Fig. 2 illustrates some relighting effect of our method 
on the Yale face Database B (The other 9 person's faces 
are used to compute the 63 ratio-images for 63 light 
sources). 




Figure2. Ratio-image based face relighting 
4. Virtual View Generation For Different Poses 

Similar to face relighting, the virtual view generation 
problem is defined as: given a frontal view of au unknown 
face, generating its view under other poses. Let P„ denote 
the frontal pose, P, be another pose (e.g. right rotating 30° 
out of Ihe image plane), and J 0 be the image under pose P„. 
Our goal is generating its view /, under pose P,. 

A linear regression method is exploited to solve this 
problem: a learning set containing pairs of Ihe shapes of 
the two views under P„ and P, is collected, a linear 
mapping between them are teamed and applied to any 
given novel frontal image to predict its shape under pose 
P,. Let ^={(/ 1 ° > / 1 1 ),(/ 3 °,/ 3 l ) > -,(/:,/:))} be an 
image set containing pairs of the two views under /^and 
P,. and Z = \(.S?,SlUS$,S\),---,(S° w Xm be ihe 
corresponding shape set containing pairs of the shapes for 
the two views in the learning set 31 . A linear mapping P 
can be leamt easily from 3 . So, for a given novel image 
/„ under pose, its shape vector 5° is first extracted using 
method in Section 2. Then, its face shape S' viewed under 
pose Pi is predicted by: 

S ] =PS°. 

Then we can generate ihe virtual view by an image 
warping procedure based on 5" and S 1 . Fig.3 shows two 
examples of the generation results, in which the first row 




Fignre 3. Virtual view generation 



5. FSS-based Face Recognition 

In our previous work, we have proposed a Face-Specific 
Subspace (FSS) based face recognition method [26]. This 
method is motivated, but essentially different from the 
traditional Eigenface. In Eigenface, each face image is 
represented as a point in a low dimensional face subspace 
shared by all faces; however, we experimentally show that 
one of the demerits of such a strategy is that the most 
discriminant features of a specific face are not accurately 
represented. Therefore, we propose to model each face by 
one individual face subspace, named Face-Specific 
Subspace. Distance from the face-specific subspace, that is, 
the reconstruction error, is then exploited as the similarity' 
measurement for identification. 

Each FSS is leamt from the training images of the 
specific face and represented as a 4-tuple by: 

% = (U i ,V k ,A l ,d k ), 
where U t is the eigenvector matrix. A, is the eigenvalues, 
<r* : is the mean of the # th face, and d k is the dimension of 
the FSS. 

Similar lo DFFS in Eigenface method, the similarity of 
any image to a face can be measured by using the Distance 
From FSS (DFFSS): less DFFSS means more probability 
that the image belongs to the corresponding face. It can be 
formulated as follows: Let rbe any input image. It can be 
projected to the A' h FSS by: W ik) = [//<&"» , where 
<&<*' =r-V k . Then can be reconstructed 

by: *<*' =t/,0"*' . So, r 's distance from k* FSS 
(DFFSS) is computed as the following reconstruction error: 
E ,f » -*«"|. 
The DFFSS can be regarded as the similarity of the 
input pattern r to the face corresponding to the FSS. 
Therefore, the following minimal distance classifier can be 
naturally formulated: 
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re£2 m if f < " , =min{e <t >}. 

ISiS/l ' 

6. Experiments on Yale Face Database B 
To evaluate the effect of augmenting training set for 
face recognition, we conduct experiments on the Yale 
Face Database B (Refer [15] for detailed information on 
this race database). We choose just the frontal set in this 
DB, containing 640 images from 10 persons, each person 
has 64 frontal images under 64 different lighting 
conditions. To test different face recognition methods, we 
choose the frontal face image under the standard lighting 
of each person as training images, other 63 images lighted 
under different for testing. 

Leave-one-out strategy is exploited to generate the 
ratio-image for each lighting configure. Then, 63 
additional face images from each standard lighting face 
images are generated to augment the training set for the 
FSS method. The testing results on the 5 subsets are 
shown in Fig. 4.{Note: other methods tested are correlation. 
PCA and FaceM.O system. They do not using the 
virtual images for train). 




Figure 4. Performance comparison on the 5 subsets in 
the Yale Face Database B 

From Fig.4, a significant performance improved can be 
observed, which obviously profits from the augment of the 
training set. 
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Online Fingerprint Template Improvement 

Xudong Jiang and Wee Ser, Senior Member, IEEE 

t^l'iTrJ,^ W ° rk P r °P° 5e , 3 a technique that Improves fingerprint templates by 
merging anr averaging mmutiae of multiple fingerprints. The weighted averaging 
fhe *rZ ^ m ^'°l!f 6 " J * Nnewrth changes of 

„!. .„ ?™A ma9 ' n9 00 .™<"ions. The recursive nature of the algorithm greatly 



during the day-to-day norma) operation. Such online template 
improving functions work by merging the input data into the 
template database during the actual application of the fmeerprinl 
verification system. r 

This paper proposes an online fingerprint template improve- 
ment algorithm with which spurious minutiae can be removed 
dropped minutiae be recovered, and wrongly labeled minutia type 
be corrected. The proposed algorithm works online during the 
mts of this technique. As a result, day-to-day operation of the fingeiprint verification system. As a 

Can be narfnrmAri nnltno Hurlnn rpsnlK I1C1M-C txriU finri t-U~ 1 i .... 



2 Minutia Set Estimation from Multiple Minutia 
Sets 



ia set. template Improvement. 



1 Introduction 

An automatic fingerprint verification system matches fingerprint 
inputs with prestored fingerprint templates, each of which consists 
of a set of features extracted from a fingerprint image. Since the most 
reliable feature for fingeiprint matching is the minutia, most current 
automatic fingerprint verification systems are based on minutia 
matching. A fingerprint template of such systems is thus a minutia 
set. The two most prominent kinds of minutiae are ridge ending and 
ridge bifurcation, which can be extracted using techniques such as 
those proposed in [1], [2], [3], [11]. Unfortunately, noise/inadequate 
0,1 1,1 111 11 »mag a ,l iti >n irt acts oft n r ii idwl 
minutia extraction very difficult The resulting undesirable results 
include spurious minutiae being produced, valid minutiae being 
osl, and the minutia type fending or bifurcation) being wrongly 
labeled. The employment of various image enhancement techniques 
l| ir[J ' J<J '<< " > J problems to a Umited extent since they 
°P erate c"'y on .i ingle fingerprint image. Maio and Maltoni [6] 
1 1 ' 1 J ' ! ' 1 1 17 " i "uttia extraction techniques \7), [8], [9], 
[10], and compared their performances. The best technique in their 
experiment produced 8.52 percent spurious minutiae, lost 4 51 per- 
cent genuine minutiae, and caused die type labeling error for 
13.03 percent minutiae, resulting in a total error of 26.07 percent. For 
the other approaches, the total errors were 33.83 percent, 119.80 per- 
cent, 207.52 percent, and 216.79 percent, respectively. From this 
experiment, we can see that perfect minutia extraction from a single 
fingerprint image is a very difficult task. 



, the image outputted from a fingerprint 
sensor has to be segmented into the background (invalid fingerprint 
region) and the valid fingeiprint region that usually covers only a 
part of a finger. Thus, different fingerprint images captured from the 
same finger usually have different (valid) fingerprint regions. A 
fingerprint region can be represented by a point set, which contains 
x- andy-coordinates of all pixels within this region, Suppose mat we 
have M fingerprint images captured from the same finger and 
obtained M minutia sets F"' = and fingerprint regions S m by 
applying a minutia extraction algorithm [111 where 



(1) 



Whereas the improvem 



fingerprint image is limited, multiple fingerprint images captured^ 
different times can be used to achieve more significant improve- 
ments since the imaging conditions thatcause theminutia extraction 
error change with time due to the changes in skin condition, climate, 
and on-site environment. However, improving the image quality 
based on multiple fingerprints is unfeasible due to the high memory 
and computation consumption required by processing multiple 
images that are not rotation and translation invariant. Instead, it is 
more feasible to improve the template minutia set by using multiple 
mmutia sets of fingerprints captured at different times. If this 
templa te improving process requires only a small memory space and 
-""ipufation time, it can be performed online in a fingerprint 
actives fingerprint inputs of users 



is a parameter vector describing the location (*<•■ , ,/>•), the direction 
±i and the type t£' of minutia k in fingerprint m. Although the 
position and direction of a finger on the sensor is u suallj diffei er I 
for the various acquisitions, pose transformation can be performed 
using a minutia matching program [12] in order for the minutia 
sets and fingerprint regions of different images to be aligned By 
matching the minutia sets, we can determine whether two 
minutiae from two different minutia sets are matched Without 
losing generality, we assume that all minutia sets F" and 
fingerprint regions S'" have been aligned, i.e., they are invariant 
to the rotation and translation of the finger, and minutiae in 
different sets have the same index k if and only if they are matched. 

For a particular physical minutia, we obtain Af sample 
measurements of its parameter vector from M' different finger- 
prints (Af < Afj. Our task is to estimate an optimal parameter 
vector based on these M' measurements, i.e., learning from 
single samples of experimental data. Tins problem could be approached 



verification system, which 
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i the context of mmimizing a suitable cost function. If the cost 
function is chosen to be the negative logarithm of the likelihood 
function derived from the sample data, this becomes equivalent to 
maximum likelihood (ML) learning. By considering a general- 
ization of the Gaussian distribution of the d (a w 1 a o ta t 
variance, the ML approach leads to a cost function of the form 

known as the Minkowski-!? error [13], where F[ is the optimal 
representative minutia parameter vector to be estimated. When the 
distribution of the data is assumed to be standard Gaussian, i.e., 
ft = 2, the cost function reduces to the sum-of-squares error' If a 
Laplacian distribution is assumed, i.e., ft = 1, the cost function 
becomes thecity block metric [13]. One problem of the standard sum- 
of-squares error is that the solution can be dominated by outliers if 
the distribution of data has heavy tails [1.4]. The use of the city block 
metric (ft. = 1) reduces the sensitivity to outliers but muiimizing it 
leads to a median operation [13] on the acquired data. This is more 
intensive to compute compared with the simple mean operation that 
minimizes the sum-of-squares error. In our context of fingapiint 
template improvement, there are really no significant data outliers 
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since minutiae with large measurement errors cannot be matched 
Willi oilier corresponding minutiae with small measuremen terrors 
TViis provides the motivation in using the sum-of-squares error as the 
cost function. 1 

Although the biological characteristics of fin R e,-prints ensure 
minutia features to be permanent and unchanging for a given finger 
[3], acquisition of minutiae information is affected by the skin and 
imagmgcondihonsatthehmeofmeaiairementandtheexactinanner 
the tinger was making contact with the sensor. As a result the 
measured minutia parameter inevitably changes with time and the 
measirremaits If can thus be seenas a temporal sequence of data. 
As such, the machine-learning task could be viewed as a problem of 
regress.on estimation, i.e., function or model learning. There axe a 
number of well-developed approaches in the literature for these 
problems, for example, LPC [IS], Kalman filtering [16], Hidden 
Markov model [17], MCMC methods [18] and EM-C algorithm [19] 
However, changes to minutia parameters may occur abruptly with 
these change,? being maintained for quite a long time due to the skin 
nature and human's habits, If is thus typically an abrupt rather than 
a sm<>oth function of m, which makes it difficult to apply the above- 
mentioned function learning approaches. Furthermore," fingerprint 
samples for a particular user are not collectedin even time intervals- 
duration between two subsequent presentations of a finger to the 
system may vary between several minutes to several months This 
again makes the above-mentioned approaches unsuitable 

Having considered the above factors and the computational 
efficiency required for an online application, we employ the 
weighted least-squares with predetermined weights as the learning 
luleTheweightsarecnosenbasedonthenanueofnrinutiasetseiies 
the objective of the integration of the multiple minutia sets and the 
computation efficiency. For instance, a higher weight should be 
nod « t gistered template than (he query fingerprint 
received m the verification process since the original template 
obtained during the registration phase is generally more reliable 
than the input minutia sets obtained during Hie day-to-day 
venJ n process. More recent erprinl inputs should also I 
assign, d u,.h higher weights than earlier ones since the integration 
of the multiple rmnuria sets is aimed at increasing the reliability of 
future matching process. The weights will be chosen in the next 
section based on these desired factors and the computational 
resources required. 

The estimation errors for all minutiae k of all minutia sets m are 
expressed as 
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offline performed by enrolling several fingerprints for each finger 
and I there is no point in online synthesizing the templates. The 
problem of the template synthesis was addressed in [20], This workis 
not aimed at solving the problem that the template represents only a 
partial fingerprint, but aimed at improving the quality of the 
template online, i.e., reducing the minutia extraction error. Thus our 
estimated template is restricted to an estimated fingerprint region 
S which can be chosen to be one of the M original fingerprint 
regions 8- or be determined by thesynthesized fingerprintregion in 
case the template synthesis is performed beforehand in the 
registration phase. 

_ Our estimated minutia set F" = {^| , tf) e S ") contains all 
minutiae in the region S' extracted from die M fingerprints As a 
result, genuine minutiae that are not extracted from some finger- 
prints can be recovered in the estimated minutia set F p if they are 
successfully extracted from some other fingerprints. However; any 
spurious minutia extracted from any fingerprint is also transfen-ed to 
the estimated minutia set if it is located within S". Therefore a 
technique lias to be developed to identify the spurious minutiae of 
the estimated templa te F p 

If an estimated minutia k is successfully extracted from 
fingerprint m, a certainty level c? = I is defined. If this minutia 
fails to lie extracted from fingerprint m but its location is within 
this fingerprint region, a certainty level 4" = 0 is defined 
However, if the region of fingerprint m does not cover this 
minutia, no irrforrnatioii about the reliability of minutia k is 
provided by fingerprint m. Thus, the reliability of each estimated 
nunufaa is described by M certainty levels defined by 

<■ = 

(I. ifi7'€F*" 
0. if^jiF^A^^eSP, ( 5 ) 

unknown, if (if . ;/£ ) g S'" 
For V ft./* eF*m = l,2,...Jl/ 1 
Similar to the calculation of the estimated minutia parameter in 
(4),acerrainryleveltf of the estimated minutia k can be estimated by 
the weighted average of £?■ over all fingerprints whose regions cover 
the minutia k ^ 



c)!' = F\" - fjf , for V(k, m)J=J" e F". (2) 

The estimated minutia F£ is obtained by rnwiirdzing the weighted 
sun\ ot the squared errors * 6 

E u t(4'f = E 

=*• Minimum, for fj" e F"', 

where < are predetermined weights. Based on this criterion, it is 
straightforward to obtain 

F ?= v 1 ^T ^E^^'^: forV/ £! i7 € F"'. (.1) 

Tlie above estimated minutia parameter If generally has better 
accuracy than Ff since it is a weighted arithmetic average over all 
matched minutiae. As a result, wrongly labeled minutiae type can 
be statistically corrected during the averaging process. 

If all estimated minutiae by (4) are collected in the estimated 
template, a template synthesis will be performed. If we match an 
input fingerprint with this template, we will face the problem of 
matching a partial input fingerprint with a much larger full 
fingerprint. False nonmatch rate may decrease but false match rate 
may increase simultaneously, or some matching criteria have to be 
changed by compromise. Further, template synthesis can be simply 



— r — r?r E W *C ^^^6^. (6) 

>».t«.l6S" »*fjf)rt" 

For authenticating a future input fingerprint, only the minutiae 
whose certainty levels are equal to or higher than a threshold Cv 
n<Cv< 1, will be used in the matdiing. This means that minutiae 
with certainty levels, lower than Cv are regarded as spurious 
minuhae, which must still remain in the template for the further 
temp ate improvement in the future or can be removed from the 
template toreduce the template size if furthertemplateirnprovement 
is not conducted. 

If we choose equal weights, the estimation of a minutia based 
on (4) and (6) is simplified to 

**"3E' £ fo^MreF'", ( 7 ) 



f«) 



where M,, is the number of fingerprints from which minutia k is 
extracted and A' t is the number of fingerprints- that cover the position 
of minutia k. In this case, we see that it is the minutia occurrence 
frequency that is used to recover dropped minutiae and remove 
spurious minutiae. 
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Fig. 1. ROC cuives on (a) DB1 ..a, (b) DB2 a, (c) DB3 a, and (d) DB4 a. 

3 Online Fingerprint Template Improvement 

It is not user-friendly to capture a number of fingerprints of the same 
finger atlongintervalsinfheregistration phase. However, during the 
verification operation of a fingerprint verification system, input 
fingerprints are successively received and compared with the 
templates. If an input fingerprint is successfully matched with a 
template, these two fingerprints are verified to have originated from 
the same finger. Therefore, input fingerprints can be used to improve 
the matched template online during the day-to-day operation of the 
fingerprint verification system. However, to use (4) and (6) to 
improve a template, all matched input minutia sets arid fingerprint 
regions need to be stored and an arithmetic average over all matched 
minutia sets need to be calculated for every template update. This 
requires a large storage space and significant computation time. 
However, storage space and verification time are often serious 
constraints, especially in stand-alone application. To reduce the 
storagespaceandprocessingUme requirements, weneedasimph'fied 
I i ngerprint region representation and a recursive algorithm. 

it is not difficult to simplify the fingerprint region representa- 
tion A polygon represented by a few points can be used to 
approximate a fingerprint region [211 and to determine whether a 
minutia point of another fingerprint is located within this region. It 
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is easy to prove that a point with location (x, y) is within a polygon 
represented by I points {x if Vj ), j = 1 . 2. . . . L, if and only if: 




Ajx +B jV + Cj < 0, for all j, j = 1, 2, 


...,L, (0) 



- Bfiij with 



where .4, = Vj - Vj _ u D s = x J+L - x„ and C, 
= (*,.»). 

A recursive algorithm that implements the weighted averaging 
in (4) and (6) can be derived by choosing the weights properly. As 
mentioned earlier, the finger skin and imaging condition changes 
with time and the template improvement is aimed at increasing the 
reliability of future matching processes. Thus, the more recent 
fingerprint inputs should be assigned higher weights than earlier 
ones, in addition, a fingerprint image is usually captured with much 
more caution during the registration process compared with those 
acquired during the day-to-day verification. Therefore, a higher 
weight should be assigned to tire original registered template. We 
thus choose a power series a" to weight the fingerprint sequence 
and another constant A to distinguish the weights of input 
fingerprints from that of the original template fingerprint. 

Let fif(N) denote the improved template minutia by using an 
original template minutia F{(0) and N input minutiae F'(n) of N 
input fingerprints, n- 1.2...N. where fingerprint n is received 
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improved template for matching 



earlier than fingcrprii 
rewritten as 



n + 1. Without losing generality, (4) can be 



if (A') = a'if ( 0) + A J2 " n KW ~ n), (Hi) 
witli the condition of 



(11) 

The power series coefficients <*»(« < l) weight the more recent 
fingerprint entries more heavily than earlier ones, while the 
constant A(A < 1) scales the weights of input fingerprints with 
respect to the original template. By choosing A = 1 - a it is not 
difficult to prove that condition (11) holds independent of the 
values of a and N. Thus, we can use the same value of a in (10) for 
different number of entries A T , Le., we can have 



if (A + 1) = tt^if (0) + A^VfftjV 



From (10) and (12) with A - 1 - 



is straightforward to obtain 



F k H (N + 1) = «if (,V) + (3 - + 1). 



(13) 



Equation (13) is the recursive template minutia parameter 
update formula where F('(N) is the old template minuria 
parameter- and if (A ? + l) the new template minutia parameter 
after the fingerprint verification system receives a new entry 
F k (.\ ! + 1). If an input minutia F/(A + 1) is located within the 
template fingerprint region, but there is no old template minutia 
if (A) matched with it, this input minutia will be merged into the 
template, i.e., if (AT + 1) = F/(.V+ 1). 

In a similar way, a recursive certainty level update formula can 
be easily derived from (6) as 



cf (JV + 1) = etc£(N) + (1 - <y)c' k (A' + 1), (u) 
where cj!(N) is the old certainty level of template minutia k and 
r. k (A + 1) the new certainty level of template minutia k after the 
system receives a new entry with certainty leve c£(,V + 1). 

It is worth noting that falsely matched input fingerprints will 
have an adverse effect on the template improvement process. To 
reduce the probability of this happening, an input fingerprint is 
used to update the matched template only if their matching score 
rm is higher dian a threshold Mu that is set to be larger than the 
verification threshold Mo of a fingerprint verification system. 
Furthermore, we could limit the shortest time interval Ti between 




, . , ,V , '< unimproved template 
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Fig. 3. Average n 



1 updates of a template to prevent any intentional 

repeated abuse of the template update process. This way, we can 
dimmish the negative effects of the online fingerprint template 
improvement procedure. 

The proposed online fingerprint template improvement ahjo- 
nthm is summarized as follows: 

1. Users enroll for the fingerprint verification system. For 
each enrolled fingerprint, a fingerprint region S" is 
segmented out and represented by L points and a minutia 
set {if } is extracted. A certainty level <£ = 1 is initialized 
for each extracted minutia. Template update time iu is 
initialized to be the current time U:. 

2. The fingerprint verification system waits until an input 
fingerprint is received. 

3- Segment out the fingerprint region and represent it with L 
points. Extract input minutia set {/■/}. Match it with each 
template {if \t£ > Cv). If the maxima] matching score 
m.H < Mv, reject this input and go to Step 2, otherwise 
output the corresponding finger ID. 

4. Read the current time 4c If mu < Mu (Mu > Mu) or 
(tc - iu) < Ti, go to Step 2, otherwise iu = fc. 

5. The matched template {if} is updated as follows: 

a. For all matched template minutiae, if and cf are 
updated by 



"if + (1 - a)F[ =*■ F k 
and at* + 1 - a => t$. 

b. Find all unmatched template minutiae located within 
the input fingerprint region by using (9). The certainty 
levels of these minutiae are updated by aef =4- c*\ 

c. The certainty levels of other template minutiae '(i e., 
those located outside the input fingerprint region) are 
unchanged, i.e., r/ k =* «£. 

cl. Find all unmatched input minutiae located within the 
template fingerprint region by using (9). Merge these 
minutiae into the template, i.e., Bi =*• Ft with 
cf = 1 - ft. 

e. Remove all template minutiae whose certainty levels 
are lower than a threshold Cu(0 < Cu < 1 - « < Cv < 
1) to limit the enlargement of the template size. Store 
the updated template and go to Step 2. 
The above proposed online fingerprint template improvement 
algorithm updates a fingerprint template using a recursive algo- 
rithm that implements a weighted averaging over all matched 
nr^erprmts.Itmcreasestheprecu'iionofnTinutia parameter corrects 
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Fig. 4. (a) False match rate and falsa nonmatch rate versus threshold and (b) RO 

wrongly labeled minutia type, recovers dropped minutiae (from 
c',' < Cv to of > Cv) and removes spurious minutiae (from if > Cv 
to r:f < Cv). Furthermore, the algoridim accords greater importance 
to more recently acquired fingerprints so tha t it weakens the effect of 
older skin and imaging conditions while strengthening recent ones. 
The recursive algorithm works only on the current input fingerprint 
and the stored template. This greatly reduces the processing time 
and storage space requirements and, therefore, enables our 
proposed approach to be online employed. 



4 Experimental Studies 

For a meaningful performance evaluation of our online fingerprint 
template improvement algorithm, the test database should contain 
not only a large number of finger IDs, but also a large number of 
sample fingerprints per finger. Unfortunately, it is very difficult to 
find or build up such a database. Thus, we decided to conduct 
experiments with two different kinds of databases. The first 
experiment used the FVC2000 [22] databases that contain a large 
number of finger IDs and eight sample fingerprints per finger. The 
second experiment used a database collected by us that contains 
only 12 finger IDs but 200 sample fingerprints per finger. The 
algorithm parameters chosen for both experiments were: L — 8, 
a = 0.8, C„ = 0.15, C, = 0.5, Mu = Mv = 0.25, and Ti = 0, 

There are four databases, DBl_a, DB2_a, DB3_a, and DB4_a, used 
in FVC2000. Each database contains 100 finger IDs and eight 
fingerprints per finger (800 fingerprints in all). Let F- denote the 
minutia set exbvicted from the ith sample fingerprint of the jth finger, 
t = 1, 2, . . . 8, j = 1. 2. . . . 100. To test the verification performance 
with the unimproved template, each template T'j = P ( was matched 
against the inputs Fj (i < k < 8) to obtain the genuine tnatelmig 
scores and each template T} = Fj was matched against the inputs 
F£ U < k < 100) to obtain the impostor matching scores. Let Tj(fc) 
denote the improved template by using the original template F ; and 
six inputs F? (1 < m < 8,m / i,m ?k,i<k<»). To test the 
verification, performance with the unproved template, each im- 
proved template T/k) was matched against the input Fjf to obtain 
the genuine matching score and each template Tj(2) was matched 
against tlie inputs Fj (J < k < 100) to obtain the impostor matching 
scores. In both tests, a total of ((S x. 7)/2) x 100 = 2,800 genuine 
matches and (100 x 99)/2 — 4,950 impostor matches were per- 
formed on each database. These numbers of matches are the same as 
that used in the FVC2000 [22]. Kg. 1 illustrates the ROC curves on the 
four FVC2000 databases. These ROC curves on the four different 
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: curves on DB_T. 

databases consistently show that our template improvement algo- 
rithm causes a significant improvement in the verification acctuacy. 

In the second experiment, a Veridicom CMOS sensor of size 
300 x 300 pixels was used to capture fingerprints. Twelve un- 
trained users were enrolled by capturing one template fingerprint 
per user. Each of these 12 users was asked to represent the enrolled 
finger to the system to produce input fingerprints several times (no 
more than five times) every working day until 199 input 
fingerprints per user were received. Each input fingerprint was 
matched online with the 12 templates and used to update (online 
improve) tire template that had the maximal matching score if this 
matching score was higher than Mu. In this way, 2,400 fingerprints 
were collected in database DB_T and the numbers of genuine 
matches and imposter matches are 199x 12 = 2,388 and 
199 x 11 x 12 = 20, 208, respectively. 

To show the template improvement progress clearly against die 
fingerprint input sequence, we averaged tlie results over the 
12 users. Fig. 2 plots the average numbers of minutiae of the input 
fingerprints, the original templates, and the improved templates as 
well as the average numbers of improved template rninutiae that 
were valid for matching. From Fig. 2, the improved template size 
increased with the first few fingerprint inputs and then stabilized 
at around twice the original unimproved template size. However, 
Fig. 2 also shows that the improved template had averaged fewer 
valid minutiae for matching than the original template, ft means 
that, ill most cases, the template improvement process removed 
more spurious minutiae compared with recovering dropped 
rrunutiae. This is because our minutia extraction algorithm, like 
most other minutia extraction approaches [6], usually produces 
more spurious rninutiae than dropped minutiae. This experiment 
also bells us that Uie improved verification performance is not due 
to the increased number of minutiae in the improved template but 
the improved template quality. 

Fig. 3 illustrates the average genuine matching scores mag and 
the average imposter matching scores nuii against the fingerprint 
input sequence. Fig. 3 clearly shows that the improved template 
produced not only higher genuine matching scores but also lower 
imposter matching scores than the unimproved template. Ob- 
viously, the higher genuine matching score is due to the template 
improvement process. Tlie lower imposter matching score also 
does not surprise us as tlie template improvement process reduced 
the number of spurious minutiae and, thus, lowered the impostor 
matciiirtg score. Both the higher genuine matching and the lower 
imposter matching scores improved the fingerprint verification 
accuracy, as shown in Fig. 4. 
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The average time taken by the fingerprint verification process 
(one minutia set extraction and one matching) was 0.147 seconds 
and the average time taken by the template improvement process 
was only 0.0003 seconds (for a Pentium 111-733 MHz PC). Our 
template improvement algorithm only decelerates tlie fingerprint 
verification system by a negligible 0.2 percent. 

Fig. 5a shows an original template minutia set- while Kg. 5b 
shows the improved template minutia set (valid for matching, i.e., 
£ 0.5}), generated using the template minutia set in Fig. 5a 
and 26 other input minutia sets. In these two figures, white dots 
represent endings, dark dots represent bifurcations, while white 
short lines representthe minutia directions. Thereare30 minutiae in 
Fig. 5a and 32 minutiae in Fig. 5b. After the template improvement, 
five spurious minutiae were removed (see circles in Ftg. 5a), seven 
dropped minutiae were recovered (see circles in Fig. 5b) and four 
minutiae had their type relabeled (see the arrows in Fig. 5a). 

5 Conclusions 

In this work, an online fingerprint template improvement algorithm 
is proposed. The proposed algorithm improves the reliability of a 
fingerprint template by using weighted averaging over all matched 
finga-prints that a fingerprint verification system receives. It reduces 
minutia extraction errors, such as Spurious minutiae, dropped 
minutiae, and wrongly labeled minutia type, which are difficult to 
avoid using only a single fingerprint image. Furthermore, the 
template is gradually changed to reflect changes in the finger skin 
and imaging conditions by weighting recent fingerprints more 
heavily. A recursive algorithm rmrdrriizes the storage space and 
computation requirements of the template improvement process. As 
a result, the proposed fingerprint template improvement process can 
beperforniedcnilineduringmeday-to-daycperah'onofafmgerprmt 
verification system. Extensive experimental studies demonstrate 
that the proposed online template improvement technique signifi- 
cantly increase the verification accuracy at a negligible cost in time. 
One problem of this technique is the enlargement of the template 
size. However, the improved template size is well limited to within 
twice the size of the original template. 
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Abstract ~ ~ 
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Fig 1. Ifltn.-cl.iss variation in fingerprints, (a) Two i^™™ 
oti the same finger separated by a period of 6 weeks exhibiting 
difference m moisture content, (b) Two impressions of the some 
finger acquired using different sensors (solid-state and optical), (c) 
Two impressions of a fingerprint exhibiting partial overlap. 



from those obtained durma authentication (see Figs. 1-3), 
resulting in an inferior performance (higher false rejects) of 
the biometric system. 

In order to account for the above variations, multiple tem- 
plates, that best represent the variability associated with a 
user's biometric data, should be stored in the database. For 
example, one could store multiple impressions pertaining to 
different portions of a user's fingerprint in order to deal with 
the problem of partially overlapping fkgerprints. Similarly, 
a user's lace image acquired from multiple viewpoints may 
be stored in order to account for variations in a person's 
pose. There is a tradeoff between the number of templates. 
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and the storage and computational overheads introduced by 
multiple templates. For an efficient functioning of a biomet- 
ric system, the process of template selection has to be au- 
tomated. However, there is limited literature dealing with 
the problem of automatic template selection in a biometric 
system. 1 In this paper we propose techniques to perform 
automatic template selection. The methods presented here 
attempt to represent the variability as well as the tvpicalitv 
observed in a user's biometric data. The proposed "methods 
have also been utilized to perform automatic template up- 
date. Our experimental results indicate the importance of 
adopting a formal procedure to perform template .selection 
and update. Although we consider a fingerprint-based bio- 
metric system as our test-bed, the techniques presented in 
this paper may be applied to other types of biometric traits 
(such as face and hand geometry) as well. 

The rest of the paper is organized as follows. In Section 
2 the two methods used to perform template selection have 
been described; in Section 3 the methodologies used to per- 
form template update have been explained: Section 4 de- 
scribes the experiments conducted to study the effectiveness 
ot the proposed techniques; Section 5 summarizes the results 
of this work and provides future directions for research. 



2. Template selection 

The problem of template selection with regard to finger- 
prints may be posed as follows: Given a set of TV fingerprint 
images corresponding to a single finger, select K templates 
that 'best' represent the variability as well as the typicality 
observed in the N images. A' <N. Currently, we assume 
that the value of K is predetermined. This systematic selec- 
tion of templates is expected to result in a better performance 
of a fingerprint matching system compared to a random se- 
lection of K templates out of the N images. 

It is important to note that template selection is different 
from template update. The term template update is used to 
refer to one of the following situations: (i ) Template aging: 
Certain biometric traits of an individual vary with age. The 
hand geometry of a child, for example, changes rapidly dur- 
ing the initial years of growth. To account for such changes, 
old templates have to be regularly replaced/augmented with 
newer ones. The old templates are said to undergo aging 
(ii) Template improvement: A previously existing template 
may be modified to include information obtained at a more 
recent time instance. For example, minutiae points may be 
added to, or deleted/modified from the template of a finger- 
print, based on information observed in recently acquired 
nnpressions [3-5]. As another example, Liu et al. [6] update 
the eigenspace in a face recognition system via decay pa- 
rameters that control the influence of old and new training 
samples of face images. Thus, template selection refers to 

1 Template selection has been sludied in the context of other 
pattern recognition problems (see Rcf. [21, for example). 
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the process by which prototype templates are chosen I'rom 
a given set of samples, whereas template update reiers to 
the process by which existing templates ore either replaced 
or modified. In the next section, we present techniques for 
template update, based on the template selection methods 
presented in this section. 

^ We propose the following two methods for template se- 

Methocl 1 ( DEND ): In this method, the N fingerprinl im- 
pressions corresponding to a user are grouped into K clus- 



ters, such thai impressions within a cluster are more similar 
than impressions from different clusters. Then for each clus- 
ter, a prototype (representative) impression that typifies the 
members of that cluster is chosen, resulting in K template 
impressions. This technique, therefore, selects prototypes 
that represent the variability observed in the impressions. 

To perform clustering, it is required to compute the 
(dis)similarity between fingerprint impressions. This mea- 
sure of ( dissimilarity is obtained by matching the minutiae 
point sets of the fingerprint impressions. Our matching 
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ulgonlhm is bused on an elastic siring matching technique 
[7], and it outputs a distance score indicating the dissimi- 
larity of the minutiae sets being compared. We use a simple 
matching algorithm since our goal is to perform template 
selection, regardless of the characteristics of the matching 
algorithm. 

Since our representation of the N fingerprint impressions 
is in the form of a N x N dissimilarity matrix instead of 
a N x d partem matrix (rf is the number of features), we 
use hierarchical clustering [8]. In particular, we use an ag- 
glomerate complete link clustering algorithm. The output 
ot tins algorithm is a dendrogram which is a binary tree, 
where each terminal node corresponds to a fingerprint im- 
pression, and the intermediate nodes indicate the formation 
ot clusters (see Fig. 4). 
The template set T, \T\ = K, is selected as follows: 
Step I : Generate the yV xN dissimilarity matrix, M where 
entry (i,j) t 1,7 G { 1 , 2, ...,//} i s the distance score between 
impressions / and J. 

Step 2: Apply the complete link clustering algorithm on 
M, and generate the dendrogram, D. Use the dendrogram D 
to identify K clusters. 

Step 3: In each of the clusters identified in step 2, select a 
fingerprint impression whose average distance from the rest 
ot the impressions in the cluster is minimum. If a cluster has 
only 2 impressions, choose any one of the two impressions 
at random. 



Step 4: The impressions selected in step 3 constitute the 
template set T. 

In Step 2, the algorithm automatically determines the 
threshold distance to cut the dendrogram and identify ex- 
actly AT clusters. For example, for the dendrogam given in 
Fig. 4, this distance is determined to be 644. We refer to 
the above algorithm as DEND since it uses the dendrogram to 
choose the representative templates. The algorithm selects 
prototypes that represent the variabilitv observed in a user's 
data. Therefore, this algorithm is prone to selecting outliers. 

Method 2 (MDIST): The second method sorts the fin- 
gerprint impressions based on their average distance score 
with other impressions, and selects those impressions that 
correspond to the K smallest average distance scores. Here, 
the rationale is to select templates that exhibit maximum 
similarity with the other impressions and, hence, represent 
typical data measurements. We refer to this method as 
MDIST since templates are chosen using a minimum dis- 
tance criteria. The prototype set selected by this technique 
represents data that is likely to occur frequently. Thus, for 

Step 1: Find the pair-wise distance score between the N 
impressions. 

Step 2: For the jth impression, compute its average dis- 
tance score, d h with respect to the other (jV- 1 ) impressions. 

Step 3: Choose impressions that have the smallest av- 
erage distance scores. These constitute the template set T, 
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The choice ibr (he value of AT is application dependent. 
Larger A' values would mean storing more templates per 
user, and ibis may nol be feasible in systems with lim- 
ited storage capacities. Moreover, in an identification sys- 
tem, matching a query (input) dala with a large number of 
templates per user would be computationally demanding. 
Smaller K values, on the other hand, may not sufficiently 
capture the intra-class variability nor the typicality of the 
data, leading to inferior matching performance. Therefore, 
a reasonable value of A', that takes into account the afore- 
mentioned factors, has to be specilied. 



3. Template update 

The methods presented in the previous section can be 
applied periodically in order to update the currently selected 
template set. In a biometric system, a user provides biometric 
dala every lime the authentication stage is invoked. Thus, 
newer samples of a user's biometric are made available over 
a period of lime. This newly acquired dala can be used 
to refresh the current template set in order to account for 
temporal changes that may occur in a person's biometric 
twit. We suggest two simple methods to perform template 
update using the newly acquired data. 

In the first method, all curreul templates are replaced with 
templates selected from the newly acquired data set, thereby 
capturing temporal changes in the fingerprint ( g chan 
due to environmental conditions, sensor characteristics or 
subject's occupational characteristics). We call this method 
BATCH-UPDATE since the previously selected batch is dis- 
carded and only the newly acquired data is considered. 

In the second method, both the current template set and 
the newly obtained data set are considered when perform- 
ing template update. The template selection procedure is 
applied after augmenting the new data set with the current 
template set. This method is called AUGMENT-UPDATE. We 
provide experimental results pertaining to both these update 
methodologies in the next section. 

The template selection and update procedure can be in- 
voked on each user independently and is not affected by the 
variable number of biometric samples available for a user. 
Moreover, it is an oil-line process that does 1101 interfere wilh 
the real-time performance of a system. As a result, the pro- 
cedure can be easily scaled to accommodate a large number 



4. Experimental results 

In order to study the effect of automatic template selec- 
tion and update on fingerprint matching, it is necessary to 
acquire several impressions per finger over a period of lime. 
Standard fingerprint databases (e.g., FVC 2002 [9]) do not 
contain a large number of impressions per finger. Therefore, 
we collected 200 impressions each of 50 different fingers in 



our laboratory using the Identfx BioTouch USB 200 optical 
sensor (255 x 256 images. 380 dpi). The data was acquired 
over a period of approximately four months with >i^ more 
than 5 impressions of a finger per day. The 200 impres- 
sions of each finger were partitioned into two sets: the first 
100 impressions (DATA1 ) were used to conduct the template 
selection experiments while the remaining 100 impressions 
(DATA2) were used in the template update experiments. Each 
of these two sets were further divided into (raining (first 
25 impressions) and test (remaining 75 pes ) :ets. 
These individual partitions were labelled as TRAIN1 TEST1 
TRAIN2 and TEST2. 



4.1. Template selection 

The template selection experiments were conducted us- 
ing DATA1. The selection procedure was applied to images 
in TRAIN1, while the matching performance was evaluated 
using images from TESTi. 

Fig. 4 shows the dendrogram obtained using the 25 fin- 
gerprint impressions of one finger. On setting K = 5. the 
resulting clusters and their prototypes as computed using 
the DEND algorithm are shown in Fig. 5; some clusters are 
seen to have only one member, suggesting the existence of 
outliers. The various prototypes are observed to have differ- 
ent regions of overlap with respect to the extracted minutiae 
points. The prototypes, for Die same finger, computed using 
the MDIST algorithm are shown in Fig. 6. 

In order to assess the matching performance of the pro- 
posed techniques (for K = 5), we match every image in 
TESTI (50 fingers, 75 impressions per finger) against the se- 
lected templates (5 per finger). When a test image is matched 
with the selected template set of a finger, 5 different distance 
.scores are obtained. The mean of these scores is reported as 
the final matching score. 2 Thus, we obtain 1 87, 500 match- 
ing scores (75 x 50 x 50) using the selected template sets. 
Fig. 7(a) shows the receiver operating characteristic (ROC) 
curves repnjsenting the matching performance of the tem- 
plate sets selected using both the algorithms. The equal error 
rates (EER) of DEND and MDIST are observed to be 7.42% 
and 6.62%, respectively. Now, for the 50 fingers, there are 
a total of ( ( 2 / ) ) - 1 non-selected template sets. It is com- 
putationally prohibitive to generate the matching scores and 
the ROC curves corresponding to all these permutations. 
Therefore, we chose 53,130 permutations (assuming that 
the impression indices in the template sel of all the 50 fin- 
gers is the .same) and computed their EER. The histogram 
of EER values is shown in Fig. 7(b), where the minimum, 
mean and maximum EER values are 6.12%, 7.89% and 
10.31%, respectively. In this histogram, the vertical lines in- 
dicate the EER values corresponding to the DEND and MDIST 



: Other techniques to combine matching scores cmi be found in 
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II 



Fig. 5. The cluster membership (A" = 5) for the dendrogram shown 
in Fig. 4. At most 5 members are indicated for euch cluster The 
prototype template in each cluster is marked with a thick border 
Note that the cluster in (b) has onlv one member 



algorithms. The percentage of non-selected template sets 
that have a lower EER than the template sets selected with 
the proposed methods is 30.3% and 1.6%, tor DEND and 
MDIST, respectively, thereby suggesting that systematic tem- 
plate selection is better than random 'selection. 

We see that the MDIST method of template selection leads 
to a better matching performance compared to the DEND 
method of selection. This may be attributed to the lact 
that MDIST selects a template set consisting of images that 



ext b 1 i \ m s ] i w iti other impressions: there- 
fore, the probability of them being correctly matched 
with impressions of the same linger is fairly high. On (he 
other hand, the DEND method is prone to selecting impres- 
sions that ate outliers thereby increasing the probability 
of iblse rejects. However, both methods are essential due 
lo the complementary nature of the template set thai tbev 
select. 

To further understand the differences and similarities 
between the template sets selected by the DEND and MDIST 
methods, we calculated the number of common impressions 
selected using these two methods (for £=5 ). The minimum 
average (over all 50 users) and the maximum values for this ' 
number is found to be 0, 1.54 and 4, respectively. Since, on 
the average. 1.54 out of a possible 5 impressions are common 
in the selected sets, the difference in performance between 
the two techniques stems from the remaining members of the 
respective sets. 

Table 1 lists the impressions of a finger that were se- 
lected as templates using the DEND and MDIST algorithms 
at different K values. The impression index indicates the 
acquisition time of the impressions— a lower index refer- 
ring to an earlier time instance. We see that there is no 
direct relationship between an impression index and its 
choice as a template. Fig. 8 shows the EERs of the two 
methods at different values of K. A good choice for K 
(that can balance system performance with the computa- 
tional/storage overheads) could be established by observing 
the knee point in the respective error curves (e.g., K = 5 
for DEND). However, a more formal method needs to be 
developed for determining the value of K for a specific 
application. 

4.1 Template update 

In this subsection, we report the system performance 
of the BATCH-UPDATE and AUGMENT-UPDATE methods. 
Observe that both these update techniques implicitly rely 
on the template selection procedures described earlier. 
They differ only in the choice of the data set on which 
template selection is performed. To demonstrate the im- 
portance of template update, we report the matching per- 
formance before and after the update process. We assume 
that the current template set of a finger consists of images 
from TRAIN!. 

In the BATCH-UPDATE method, template selection was 
performed on TRAH2 after completely discarding the tenv 
plate set extracted from TRAINi. In the AUGMENT-UPDATE 
method, on the other hand, images from TRAIN2 were 
first augmented with the 5 current templates, and template 
selection was performed on the augmented set. Both the 
update techniques selected 5 templates from their respec- 
tive candidate sets. The matching performance of the two 
update techniques was evaluated using data set TEST2. Fig. 
9 shows the ROC curves indicating the performance be- 
fore template update (i.e., templates selected from TRAINI 
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were tested on TEST2) and after incorporating the template 
update procedures. Table 2 lists the EERs of both the up- 
date methods based on the selection technique (DEND and 
MDIST) that was employed. It is seen that both the update 
methods result in substantial improvement in matching 
performance. 

We observe from the ROC curves and EER values 
that AUGMENT-UPDATE results in better performance than 
BATCH-UPDATE. This is because in AUGMENT-UPDATE we 
give the previously selected templates a chance to com- 
pete for reselection. This can help in retaining long-term 
trends in the characteristics of the fingerprint impressions 
leading to the observed improvement in performance hi 
AUGMENT-UPDATE, the average number of reselected tem- 
plates (the average is taken over 50 users) was found 
to be 1.5 and 0.62 using the DEND and MDIST methods 
respectively. 



5. Discussion and future work 

A systematic procedure for template selection and up- 
date is critical to the performance of a biometric system. 
In tins paper we have proposed two techniques to perform 
template .selection m the context of a fingerprint machine 
system. Both techniques are based on (he distance score 
between pairs of fingerprint impressions originating from 
the same finger. The first method called DEND utilizes a 
clustering scheme to detect prototype impressions The 
template set selected by this technique captures the vari- 
abjhly observed in a user's fingerprint image. The second 
method called MDIST ranks the fingerprint impressions 
based on their average distance from the other impressions, 
and Ihen selects impressions whose average distance is the 
leasl. Tims, il aids in selecting a template set that exhibit 
maximum similarity with the other impressions. Our ex- 
periments demonstrate that a systematic template selection 
procedure results in better performance 1han random tem- 
plate selection; il was also observed that the MDIST tech- 
nique results in better performance 1han DEND. Currently 
we are studying ways to effectively combine the two tech- 
niques in order to thither improve system performance. We 
are also considering methods to deteimine the value of K 
automatically. 

We have also proposed two template update methods that 
refresh the current template set based on newly acquired 
biometric data. The update methods implicitly rely on the 
template selection process. The AUGMENT-UPDATE technique 
performs template selection by considering both the current 




Number of templates (K) 

Fig. 8. The EER of the fingerprint matcher plotted us a function of A'. 
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False Accept Rate (%) 

fw!' ^""T !''° Wing ' m|,r0ve " lent iwfonwwce when the BATCH-UPDATE and AUGMENT-UPDATE procedure- 
(a) the DEND method ol template selection was used, while in (b) the MDIST method wn S employed. 



template set as well as the newly acquired data set. The 
BATCH-UPDATE technique, on the other band, considers only 
the newly acquired data .set and completely discards the cur- 
rent template set. Both the update methods were shown to 
improve the matching performance ol the system. Our exper- 
iments indicate that AUGMENT-UPDATE yields better match- 
ing performance than BATCH-UPDATE. 

It must be mentioned that the template selection and up- 
date techniques described here are based on the distance 



measure (scores) between pairs of fingerprint impressions. 
We have, therefore, adopted a featureless approach to clus- 
tering |11]. Hence, if a different fingerprint matching algo- 
rithm is used, a different set of prototype impressions is likely 
to be obtained. We are developing alternate techniques that 
would operate on the raw images in order to detect proto- 
type impressions. We are also in the process of testing our 
techniques on other biometric modalities as well (viz., face 
and hand geometry). 
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Tabic 2 

EER values before and after incorporating the template update 



Update technique EER (%) 

DEND method 

No update 10.61 

BATCH-UPDATE 9.55 

AUGMENT-UPDATE 7.37 

MOIST method 

No update 10.32 

BATCH-UPDATE 7.69 

AUGMENT-UPDATE 6.31 
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